Skip to content

Expose InternalServer as a transport attribute in ServerImpl#12668

Open
vlcekmilan wants to merge 1 commit intogrpc:masterfrom
vlcekmilan:expose-transport-server-attribute
Open

Expose InternalServer as a transport attribute in ServerImpl#12668
vlcekmilan wants to merge 1 commit intogrpc:masterfrom
vlcekmilan:expose-transport-server-attribute

Conversation

@vlcekmilan
Copy link

This change exposes the InternalServer as a transport-level attribute on ServerTransportListenerImpl, making it available to ServerTransportFilter implementations and downstream consumers via Attributes.

Changes

  1. Added a static Attributes.Key named TRANSPORT_SERVER_ATTR with the key "io.grpc.Grpc.TRANSPORT_ATTR_SERVER_TRANSPORT"
  2. In transportReady(), the transportServer reference is set on the Attributes before transport filters are invoked, so it is accessible throughout the request lifecycle

Why
We need access to the underlying ServerTransport from within a ServerInterceptor to force-close the transport connection when an external session expires.

In our use case, we have a ServerInterceptor that authenticates incoming RPCs against an external session management system. When a session isestablished, we register a close handler on it. If the external session is later invalidated or expires, the close handler fires and needs to shut down the underlying transport so the client is forced to reconnect and re-authenticate.

Currently, there is no way for a ServerInterceptor to access the transport layer to force-close a connection. The interceptor can reject individual RPCs via call.close(Status.UNAUTHENTICATED, ...), but it cannot terminate the transport itself.

@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Feb 26, 2026

CLA Signed

The committers listed above are authorized under a signed CLA.

  • ✅ login: vlcekmilan / name: Milan Vlcek (d49ab2f)

@vlcekmilan vlcekmilan force-pushed the expose-transport-server-attribute branch from 6e9994a to 1e53d90 Compare February 26, 2026 12:13
@vlcekmilan vlcekmilan force-pushed the expose-transport-server-attribute branch from 1e53d90 to d49ab2f Compare February 26, 2026 14:48
@vlcekmilan
Copy link
Author

Hello @ejona86, could you pls review this?

@ejona86
Copy link
Member

ejona86 commented Feb 27, 2026

exposes the InternalServer

That's internal. We're definitely not going to expose it directly. We'd have to do something indirectly.

When a session isestablished

How is a session established?

so the client is forced to reconnect and re-authenticate.

Is authentication an RPC, or part of TLS handshake? Or are you doing something with ServerTransportFilter?

@ejona86
Copy link
Member

ejona86 commented Feb 27, 2026

If the external session is later invalidated or expires

FWIW, predictable expiration could potentially be handled by maxConnectionAge(). But if sessions can be suddenly invalidated, that's quite a different thing.

@vlcekmilan
Copy link
Author

FWIW, predictable expiration could potentially be handled by maxConnectionAge(). But if sessions can be suddenly invalidated, that's quite a different thing.

The session can be terminated by the other system at any time, so its expiration cannot be predicted.
The code example is in the grpcServerInterceptorr where we need to close the transportConnection..
image

@vlcekmilan
Copy link
Author

Is authentication an RPC, or part of TLS handshake? Or are you doing something with ServerTransportFilter?

Is not part of the TLS handshake, auth happens inside grpc interceptCall(), which is invoked per RPC call, after the transportConnection is already established.
We just need to close the transportConnection when session is invalidated as shown on picture above.

@kannanjgithub
Copy link
Contributor

kannanjgithub commented Mar 2, 2026

Instead of a hard socket close, you might want to send an HTTP/2 GOAWAY. gRPC clients handle GOAWAY by transparently creating a new connection for the next request. To do that you can create a wrapper around GrpcHttp2ConnectionHandler that calls graceful close, and define an attribute key for this wrapper object:

import io.grpc.netty.GrpcHttp2ConnectionHandler;

public final class TransportControl {
    public static final Attributes.Key<TransportControl> KEY = Attributes.Key.create("transport-control");
    private final GrpcHttp2ConnectionHandler handler;

    public TransportControl(GrpcHttp2ConnectionHandler handler) {
        this.handler = handler;
    }

    public void sendGoAway() {
        // This triggers an HTTP/2 GOAWAY frame.
        // The client will see this and stop sending new requests on this connection.
        handler.getGracefulServerCloseCommand().run();
    }
}

Write a custom ProtocolNegotiator to grab the handler before it gets buried in the Netty pipeline:

public final class GoAwayNegotiator implements ProtocolNegotiator {
    private final ProtocolNegotiator delegate;

    public GoAwayNegotiator(ProtocolNegotiator delegate) {
        this.delegate = delegate;
    }

    @Override
    public Handler newHandler(GrpcHttp2ConnectionHandler grpcHandler) {
        // Inject the handler into the attributes
        Attributes capturedAttrs = Attributes.newBuilder()
                .set(TransportControl.KEY, new TransportControl(grpcHandler))
                .build();
        
        grpcHandler.addTransportArg(capturedAttrs);
        return delegate.newHandler(grpcHandler);
    }
    // ... delegate other methods (scheme, close)
}

Include the custom ProtocolNegotiator when configuring the NettyServerBuilder:

Server server = NettyServerBuilder.forPort(8080)
    .protocolNegotiator(new GoAwayNegotiator(ProtocolNegotiators.plaintext()))
    // 1. How long to wait for active RPCs to finish after GOAWAY
    .gracefulShutdownTimeout(5, TimeUnit.SECONDS) 
    // 2. Maximum time the entire server-side shutdown process can take
    .shutdownTimeout(10, TimeUnit.SECONDS)
    .addService(new MyServiceImpl())
    .build();

Add a listener for handling the external session close in your server interceptor:

public class SessionInterceptor implements ServerInterceptor {
    @Override
    public <ReqT, RespT> ServerCall.Listener<ReqT> interceptCall(
            ServerCall<ReqT, RespT> call, Metadata headers, ServerCallHandler<ReqT, RespT> next) {
        
        // Extract the controller we injected at the Netty level
        TransportControl control = call.getAttributes().get(TransportControl.KEY);

        if (control != null) {
            // Register this connection/control with your external session manager
            String sessionId = headers.get(SESSION_ID_KEY);
            sessionManager.register(sessionId, () -> {
                // When session expires, stop accepting new streams on the connection.
                control.sendGoAway(); 
            });
        }

        return next.startCall(call, headers);
    }
}

Make sure to have a clean-up strategy for dead channels that may occur because the channel has been closed from the client side, and occupy unnecessary memory until the external session expires.

public class CleanupFilter extends ServerTransportFilter {
    @Override
    public void transportTerminated(Attributes attrs) {
        // This is called when the TCP connection actually dies
        TransportControl control = attrs.get(TransportControl.KEY);
        String sessionId = attrs.get(SESSION_ID_KEY); 
        
        // Remove from your map so the GC can claim the Channel
        sessionManager.unregister(sessionId); 
        logger.info("Connection closed; cleaned up session mapping.");
    }
}

If you must close the transport instead of sending GO_AWAY, you can obtain the channel with GrpcHttp2ConnectionHandler.getChannel() and call its close in the TransportControl class above. However note that a hard channel.close() logs Connection reset by peer errors on both client and server. GOAWAY is a standard part of HTTP/2 flow control and is handled silently.

@ejona86
Copy link
Member

ejona86 commented Mar 2, 2026

Is not part of the TLS handshake, auth happens inside grpc interceptCall(), which is invoked per RPC call, after the transportConnection is already established.
We just need to close the transportConnection when session is invalidated as shown on picture above.

How does the client react to that? gRPC clients won't handle the connection close in any special way. If auth is a regular RPC, then you can handle the revocation in normal RPCs (e.g., responding UNAUTHENTICATED).

Instead of a hard socket close, you might want to send an HTTP/2 GOAWAY.

transport.shutdown() triggers graceful, double-GOAWAY shutdown (which they are showing code using). shutdownNow() is abrupt.

Write a custom ProtocolNegotiator to grab the handler before it gets buried in the Netty pipeline:

@kannanjgithub, those APIs are internal to gRPC. They should not be used by others.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants